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Introduction 


Medical  care  is  increasingly  the  responsibility  of  teams  rather  than  individuals.  This  realization 
has  led  to  increased  team  training  of  medical  professionals  with  a  focus  on  establishing  role 
appropriate  team  behaviors  emphasizing  team  communication,  leadership  and  team  member 
skills.  Emergency  departments  and  trauma  centers  are  unusual  care  settings  in  that 
circumstances  provide  little  time  for  deliberation  and  planning  at  the  time  of  patient  care  due  to 
the  emergency  nature  of  the  cases.  Furthermore,  care  is  provided  by  ad  hoc  teams  where  the 
composition  of  the  group  responsible  for  delivery  of  care  is  not  constant  but  the  roles  and 
responsibilities  needed  are  invariant.  Finally,  members  of  the  team  may  enter  at  various  points 
as  the  case  unfolds.  These  conditions  require  special  team  leadership,  communication  and 
team  member  skills.  The  purpose  of  this  research  is  to  develop  a  program  of  systematic,  brief 
training  in  role  appropriate  team  behaviors  covering  key  communication,  leadership,  and  team 
member  behaviors  for  emergency  medical  care  teams  and  to  determine  whether  training  in 
these  behaviors  will  result  in  improved  targeted  individual  and  ad  hoc  team  communication, 
leadership  and  team  member  behaviors  in  simulated  emergency  care  situations  such  as  those 
regularly  faced  by  trauma  and  emergency  department  medical  care  teams. 

During  the  second  year  we  requested  a  change  in  research  design  and  a  no-cost  extension  of 
the  project.  This  request  for  a  research  design  change  and  an  extension  was  granted  on  21-Jul 
201 1 .  As  a  result,  the  revised  date  for  completion  of  the  project  was  changed  to  30  Dec  2012. 
Figure  1  summarizes  the  approved  modified  design  of  the  study.  In  the  body  of  this  final  report 
we  provide  a  description  of  project  accomplishments  broken  down  by  the  tasks  and  subtasks 
spelled  out  in  the  approved  statement  of  work  as  reflected  in  the  original  proposal  and  in  the 
approved  request  for  a  no-cost  extension. 
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Figure  1  summarizes  the  modified  design  of  the  study. 

Figure  1.  Study  Design 
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Body 


Task  1 .  Develop  the  training  module 

a.  Conduct  an  in-depth  analysis  of  in-situ  simulation  results 

An  in-situ  simulation  of  a  trauma  case  was  performed  at  Memorial  Medical  Center  in 
July  2008  to  test  staff  preparedness  for  addressing  trauma  cases  and  to  uncover 
weaknesses  that  needed  to  be  addressed.  The  results  of  this  in-situ  simulation  were 
a  stimulus  for  the  current  project  and  provided  guidance  for  the  proposal  objectives. 
Core  Training  Planning  and  Development  Group  performed  a  more  in-depth  analysis 
of  the  in-situ  simulation  as  a  way  of  validating  and  refining  the  proposed  primary 
focus  for  the  planned  training  interventions.  This  group  reaffirmed  the  weaknesses 
identified  in  the  project  proposal.  Further  the  group  concluded  that  a  primary 
overarching  problem  was  that  the  residents  on  the  trauma  team  appeared  to  lack  a 
clear  understanding  of  their  roles  and  that,  during  trauma  cases,  resident  leadership 
of  the  team  was  never  clearly  established. 


b.  Conduct  additional  observation  of  emergency  and  trauma  cases  to  establish 
the  representativeness  of  in-situ  findings  and  to  provide  additional  information 
as  needed 

Project  Investigators  and  the  Core  Training  Planning  and  Development  Group 
developed  a  role  appropriate  team  behavior,  communication  and  leadership  checklist 
for  use  in  observing  trauma  care  at  Memorial  Medical  Center.  This  checklist  was 
developed  with  the  assistance  of  physicians  who  work  in  the  emergency  department 
and  trauma  center  at  Memorial  Medical  Center.  It  was  designed  to  be  directly  aligned 
with  the  planned  target  training  behaviors  developed  for  this  project.  In  February 
2010,  members  of  the  research  team  observed  15  trauma  cases  and  completed  the 
communication  and  leadership  checklist  for  each  case.  The  goal  of  this  observation 
was  to  determine  whether  or  not  what  we  observed  in  the  in-situ  simulation  occurred 
regularly  in  actual  patient  care  situations.  All  members  of  the  trauma  and  ED  teams 
were  notified  of  our  activities.  Two  investigators  were  given  trauma  pagers  and  ID 
badges  that  allowed  keyless  access  to  the  trauma  bay  area.  All  but  two  observed 
traumas  took  place  during  normal  work  hours.  Each  case  was  assigned  a  random 
number.  There  is  no  way  to  tie  the  random  number  to  individual  patients.  No 
information  about  patient  care  was  gathered,  nor  was  any  protected  health 
information  gathered. 

Eleven  of  the  fifteen  cases  were  observed  from  a  time  prior  to  the  patient’s  arrival 
through  the  primary  and  secondary  survey,  and  through  stabilization  and  imaging  of 
the  patient.  For  four  of  the  cases,  observation  began  after  the  patient  had  been 
delivered  to  the  trauma  bay. 
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The  following  tables  provide  observation  frequency  data  for  each  item  on  the 
Communication  and  Leadership  checklist.  Items  with  an  X  in  the  upper  left  hand  box 
are  those  where  the  standard  practices  we  observed  frequently  deviated  from  the 
desired  practices  as  delineated  either  in  the  literature  and/or  by  the  trauma/ED  team 
members  in  our  core  training  development  group. 


Did  team  members  introduce  themselves  by  role  and  by  name? 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

No 

13 

86.7 

86.7 

86.7 

Not  observed 

2 

13.3 

13.3 

100.0 

Total 

15 

100.0 

100.0 

Did  an  organized  handoff  occur  between  EMT  and  receiving  team? 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

No 

10 

66.7 

66.7 

66.7 

Yes 

4 

26.7 

26.7 

93.3 

Not  applicable/ 

1 

6.7 

6.7 

100.0 

observed 

Total 

15 

100.0 

100.0 

Did  an  organized  handoff  occur  between  Emergency  Medicine  and  Trauma 


teams? 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

No 

6 

40.0 

40.0 

40.0 

Yes 

5 

33.3 

33.3 

73.3 

Not  applicable/ 

4 

26.7 

26.7 

100.0 

observed 

Total 

15 

100.0 

100.0 

Did  an  organized  handoff  occur  between  the  Trauma  team  and  the  new 
care  team  at  final  disposition? 


Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

No 

6 

40.0 

40.0 

40.0 

Yes 

3 

20.0 

20.0 

60.0 

Not  applicable/ 

6 

40.0 

40.0 

100.0 

observed 

Total 

15 

100.0 

100.0 

Was  extraneous  noise  minimal  and/or  explicitly  controlled? 


X 

Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

No 

9 

60.0 

60.0 

60.0 

Yes 

6 

40.0 

40.0 

100.0 

Total 

15 

100.0 

100.0 

How  frequently  did  read  backs  of  orders  occur? 


X 

Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Never 

8 

53.3 

57.1 

57.1 

Rarely 

4 

26.7 

28.6 

85.7 

Some  of  the 

1 

6.7 

7.1 

92.9 

time 

Most  of  the 

1 

6.7 

7.1 

100.0 

time 

Total 

14 

93.3 

100.0 

Missing 

1 

6.7 

Total 

15 

100.0 
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Did  team  members  avoid  walking  on  the  communication  of  others? 


Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid  Never 

1 

6.7 

7.7 

7.7 

Rarely 

1 

6.7 

7.7 

15.4 

Some  of  the 

4 

26.7 

30.8 

46.2 

time 

Most  of  the 

7 

46.7 

53.8 

100.0 

time 

Total 

13 

86.7 

100.0 

Missing 

2 

13.3 

Total 

15 

100.0 

Was  all  care  communicated  and  documented? 


X 

Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid  Never 

9 

60.0 

64.3 

64.3 

Rarely 

5 

33.3 

35.7 

100.0 

Total 

14 

93.3 

100.0 

Missing 

1 

6.7 

Total 

15 

100.0 

Number  of  SMARTT  Stepbacks  that  occurred 


X 

Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid  0 

13 

86.7 

86.7 

86.7 

1 

2 

13.3 

13.3 

100.0 

Total 

15 

100.0 

100.0 
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Were  all  team  members  included  in  SMARTT  Stepbacks? 


X 

Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid  No 

13 

86.7 

86.7 

86.7 

Yes 

2 

13.3 

13.3 

100.0 

Total 

15 

100.0 

100.0 

Number  of  team  members  who  spoke  during  SMARTT  Stepbacks 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  Not  Applicable 

13 

86.7 

86.7 

86.7 

1  person  only 

2 

13.3 

13.3 

100.0 

Total 

15 

100.0 

100.0 

Number  of  team  members  who  listened  during  SMARTT  Stepbacks 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  Not  Applicable 

14 

93.3 

93.3 

93.3 

Minority,  but 

1 

6.7 

6.7 

100.0 

more  than  one 

Total 

15 

100.0 

100.0 

SMARTT  Stepback  element  1  was  present:  Current  situation  was  discussed 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

15 

100.0 

100.0 

100.0 

SMARTT  Stepback  element  2  present:  Direction  (goals)  were  discussed 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

15 

100.0 

100.0 

100.0 
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Stepback  element  3  present:  Velocity  (urgency)  was  discussed 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

15 

100.0 

100.0 

100.0 

SMARTT  Stepback  element  4  present:  Connections  (resources  needed) 

discussed 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

15 

100.0 

100.0 

100.0 

SMARTT  Stepback  element  5  present :  Turbulence  (Anticipated  Problems) 

discussed 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

15 

100.0 

100.0 

100.0 

SMARTT  Stepback  element  6  present:  Talk  to  Me  (Team  members  explicitly 

encouraged  to  speak) 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

15 

100.0 

100.0 

100.0 

Team  behavior  reflected  that  roles  of  Attending  (Supervising)  Physician  and 

Team  Leader  were  understood 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

5 

33.3 

33.3 

33.3 

Yes 

10 

66.7 

66.7 

100.0 

Total 

15 

100.0 

100.0 
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Team  leader  indicated  role  by  taking  position  at  foot  of  bed 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

15 

100.0 

100.0 

100.0 

Team  leadership  explicitly  established  (assigned  or  assumed)  and 


communicated 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

7 

46.7 

46.7 

46.7 

Yes 

7 

46.7 

46.7 

93.3 

Not  applicable 

1 

6.7 

6.7 

100.0 

Total 

15 

100.0 

100.0 

Team  leader  explicitly  assigned  and  delegated  responsibilities  to  other 


team  members 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

11 

73.3 

73.3 

73.3 

Yes 

2 

13.3 

13.3 

86.7 

Not  applicable 

2 

13.3 

13.3 

100.0 

Total 

15 

100.0 

100.0 

Transitions  in  team  leadership  were  clear  and  explicit 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

11 

73.3 

73.3 

73.3 

Not  applicable 

4 

26.7 

26.7 

100.0 

Total 

15 

100.0 

100.0 
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It  was  clear  who  the  team  leader  was  at  all  times 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

7 

46.7 

46.7 

46.7 

Yes 

8 

53.3 

53.3 

100.0 

Total 

15 

100.0 

100.0 

Team  leader  and/or  supervising  physician  managed  noise  appropriately 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

6 

40.0 

40.0 

40.0 

Yes 

6 

40.0 

40.0 

80.0 

Not  applicable 

3 

20.0 

20.0 

100.0 

Total 

15 

100.0 

100.0 

Team  leader  and/or  supervising  physician  explicitly  directed  care 


X 

Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  No 

6 

40.0 

40.0 

40.0 

Yes 

9 

60.0 

60.0 

100.0 

Total 

15 

100.0 

100.0 
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c.  Review  additional  research  reports  and  professional  literature  (e.g. 
TeamSTEPPS,  American  College  of  Surgeons,  and  Association  of  Program 
Directors  in  Surgery  Curriculum)  on  team  leadership  and  communication  in 
emergency  care  settings. 

Members  of  the  research  team  reviewed  numerous  research  reports  regarding 
leadership  and  communication  which  have  informed  the  further  refinement  of  the 
target  behaviors  which  will  make  up  the  focus  of  the  training  module.  We  also 
reviewed  recommendations  and  curricula  from  various  organizations  (Association  of 
Program  Directors  in  Surgery  Curriculum)  and  acquired  policies  and  procedures 
regarding  team  performance,  leadership,  communication  and  care  coordination  from 
various  other  hospitals  which  have  been  helpful.  We  found  the  material  provided  to 
us  by  Vanderbilt  University  to  be  particularly  helpful  and  have  incorporated  some  of 
their  ideas  about  emergency  care  policies  and  procedures  into  our  work. 

d.  Consult  with  representatives  of  hospital  administration  and  various  health 
professional  constituencies  regarding  the  focus  of  new  training  and  its 
relationship  to  current  and  past  training. 

The  Program  Advisory  Committee  for  this  project  was  made  up  of  representatives  of 
all  affected  professional  constituencies.  The  program  advisory  committee  met 
monthly  and  has  been  briefed  regularly  on  proposed  target  training  behaviors, 
training  methods,  and  research  methods.  This  group  has  participated  and  endorsed 
all  decisions  regarding  these  project  developments. 

Based  on  the  results  reported  above,  the  Core  Training  Planning  and  Development  Group  and 
the  Program  Advisory  Committee  agreed  that  the  primary  focus  for  the  interventions  would  be 
on  training  emergency  and  trauma  team  leaders  and  members  to  communicate  more  effectively 
and  to  function  effectively  as  trauma  team  leaders  and  team  members  during  trauma 
resuscitations.  They  also  agreed  that  the  proposed  specific  target  training  behaviors  were 
reasonable  targets  to  drive  the  training  intervention  for  trauma  team  members. 

Table  1 .  provides  a  listing  of  the  team  behaviors  that  were  targeted  for  training. 

Particular  emphasis  was  placed  on  teaching  trauma  teams  to  use  SMARTT  Stepbacks. 
SMARTT  Stepbacks  are  similar  to  the  “huddle”  in  TeamSTEPPS.  The  SMARTT  Stepbacks  are 
designed  to  assure  team  shared  awareness  of  the  direction  (where  we  are  and  where  we  are 
going  with  this  patient)  velocity  or  rapidity  (How  fast  we  need  to  go),  priorities,  obstacles  and 
approaches  to  dealing  with  them,  and  anticipated  developments  (Things  to  “keep  an  eye  on”). 
They  are  also  designed  to  encourage  two-way  communication  among  team  members.  SMARTT 
Stepbacks  are  a  way  of  establishing  that  all  personnel  involved  have  the  same  expectations  for 
the  patient  outcome  and  for  task  urgency  and  priorities.  While  we  envisioned  that  SMARTT 
Stepbacks  normally  will  be  used  by  the  trauma  team  after  the  primary  survey  in  the  trauma  bay, 
SMARTT  Stepbacks  can  be  used  whenever  the  team  leader  feels  they  are  needed. 
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e.  Design  and  produce  the  necessary  training  module  and  supporting  materials 

One  training  module  was  developed  and  pilot  tested.  The  training  module  included 
all  target  behaviors  including  1)  the  SMARTT  Stepback  Communication  portion 
(situation;  management;  activity  -What  needs  to  happen  next;  rapidity  -  what 
needs  to  be  done  first  and  how  quickly;  troubleshoot  -  what  may  go  wrong  and 
steps  to  correct;  talk  to  me  -  what  are  your  concerns,  what  has  leader  missed,  what 
other  pertinent  information  needs  to  be  shared);  2)  the  leadership  elements  for 
trauma  team  leaders  (explicit  assignment  of  tasks  to  team  members  by  name  and/or 
function,  explicit  transitions  of  leadership  when  leadership  is  handed  over  to  others, 
managing  noise  in  the  room,  managing  workload  of  team  members,  encouraging 
team  members  to  volunteer  key  information)  and  3)  the  role  appropriate  team 
behaviors  for  other  team  members  (e.g.  announce  yourself  by  name  and  role  when 
entering  the  trauma  bay,  volunteer  important  information,  speak  only  when 
necessary  for  patient  care,  verbally  confirm  orders  and  report  completion  of  tasks). 

Videos  demonstrating  the  target  behaviors  were  planned  and  then  filmed  on  9  June 
201 1 .  The  demonstration  videos  were  edited  and  incorporated  into  the  Powerpoint 
presentation  used  in  the  training  intervention. 


Task  2.  Development  of  Simulation  Scenarios 

After  reviewing  the  30  simulation  cases  in  the  METI  Disaster  Medical  Readiness  and 
Emergency  Medical  Services  Learning  Modules,  cases  developed  locally  and  team  training 
cases  developed  for  the  Association  of  Program  Directors  in  Surgery  curriculum  under  the 
leadership  of  Gary  Dunnington,  MD,  at  Southern  Illinois  University  School  of  Medicine, 
members  of  the  Core  Training  Planning  and  Development  Group  developed  four  simulated 
trauma  resuscitation  scenarios  for  use  in  the  training.  This  involved  developing:  a.  the 
educational  rationale  and  the  learning  objectives  for  the  simulation  scenarios,  b.  the  case  stem 
for  the  learners  (pertinent  patient  and  scenario  information  -  task,  location,  physician/help 
availability,  equipment  availability,  c.  background  and  briefing  information  for  facilitators  and 
coordinators  (patient  data  background  and  baseline  patient  state,  baseline  simulator  state),  d. 
supporting  files  (chest  x-ray,  echo,  assessment,  handouts  etc.),  e.  a  list  of  alterations  that  will 
occur  during  the  simulation,  f.  the  trigger  to  move  to  the  next  state,  g.  the  expected  learner 
actions,  and  h.  the  teaching  points  for  the  debriefing. 

One  case  was  used  exclusively  for  demonstrating  the  target  behaviors  as  part  of  the  training 
intervention.  The  other  three  were  alternated  for  use  as  1 .)  pre-intervention  exercises  designed 
to  determine  the  rate  of  team  target  behaviors  prior  to  training,  2)  opportunities  to  practice  using 
the  target  behaviors  at  the  end  of  the  training  session  and  3)  retention  simulation  exercises  to 
document  retention  of  the  target  behaviors  approximately  three  weeks  after  training. 

The  practice  simulation  exercise  and  the  post  intervention  retention  exercise  were  followed  by 
team  debriefing  sessions  designed  to  cement  the  desired  target  behaviors  and  to  correct  target 
behaviors  performed  incorrectly.  The  debriefing  protocol  (Appendix  1)  was  designed  by  the 
research  team.  Debriefing  was  done  by  two  trauma  surgeons  (Jarrod  Wall,  MD  and  John 
Sutyak,  MD)  and  two  emergency  department  physicians  (David  Griffen,  MD  and  Christopher 
McDowell,  MD)  who  were  trained  by  Reed  Williams,  Ph.D.  using  Appendix  1  as  a  training  aid. 


16 


Task  3.  Data  collection  and  interpretation  plan  for  analyzing  individual  and  team 
performance  data  during  simulation  scenarios 

The  plan  for  data  collection  and  interpretation  was  completed  in  the  1  March  201 1  to  31  May 
201 1  Quarter.  Details  will  become  apparent  when  reading  the  project  results  (See  Task  8) 

Task  4.  Acquire  and  install  the  simulators,  and  medical  equipment  and  supplies  needed 
to  make  the  simulation  center  resemble  as  closely  as  possible  the  trauma  bay  and  the 
emergency  department. 

All  simulation  equipment,  recording  equipment,  supporting  medical  equipment  and  supplies  was 
acquired  and  installed.  Our  goal  in  selecting  and  acquiring  simulation  and  medical  equipment 
was  to  enable  us  to  accurately  simulate  clinical  working  environments  in  both  the  simulation 
laboratory  and  the  trauma  bay  and  to  record  trainee  performance  on  simulations  for  debriefing 
purposes  and  for  documentation  of  training  effectiveness.  For  training  purposes  both  the 
simulation  laboratory  and  one  trauma  bay  were  utilized.  The  recording  equipment  acquired 
allowed  the  research  group  to  perform  an  in  depth  analysis  of  the  performance  of  the  learners 
and  evaluation  of  the  curriculum.  It  was  also  used  for  training  purposes,  demonstrating  gold 
standard  behavior  as  well  as  allowing  for  team  and  self  analysis  of  performances. 

During  the  initial  year  we  experienced  substantial  reliability  issues  with  iSTAN.  As  a  result  we 
conducted  a  thorough  investigation  of  simulators  offered  by  other  vendors  and  of  alternative 
simulators  available  through  METI.  Based  on  our  investigation  we  decided  to  substitute  the 
METI  Man  Simulator,  for  iSTAN.  We  concluded  that  the  METI  Man  Simulator:  1)  was  less 
expensive,  2)  was  less  sophisticated  but  more  durable  for  multiple  users,  and  3)  served  the 
purposes  of  this  grant  and  our  needs  better. 


Task  5.  Training  interventions  and  the  associated  simulation  scenarios  pilot  tested. 
(Milestone  Completion  Date:  31  August  2011) 

The  training  intervention  was  pilot  tested  on  9  September,  201 1  with  John  Sutyak,  MD,  Director 
of  the  Trauma  Center,  providing  the  training.  A  nurse  (Cathy  Schwind,  RN),  a  general  surgery 
resident  with  advanced  training  as  a  surgical  education  fellow  (Michael  Kim,  MD)  and  a  learning 
psychologist  (Reed  Williams,  Ph.D.)  were  present.  Suggestions  were  made  and  changes  were 
made  in  the  presentation  as  needed. 

The  three  simulation  scenarios  were  pilot  tested  on  22-23  June  201 1 .  The  pilot  testing  was 
conducted  by  two  nurse  educators  (Cathy  Schwind,  RN,  Margaret  Boehler,  RN)  two  trauma 
nurses  (Linda  Riesman,  RN,  Becky  O’Sullivan,  RN)  a  trauma  surgeon  (Christopher  Wohltmann, 
MD)  and  a  technician  from  the  simulation  center  (Chris  Reavis).  The  focus  of  the  pilot  test  was 
to  assure  that  the  simulator  functioned  effectively,  and  to  assure  that  the  simulations  were 
realistic  representations  of  actual  trauma  cases. 


Task  6  and  7.  Training  Intervention  Completed  for  All  Participants  (Milestone  Completion 
Date:  30  September  2011).  Pre,  Post  and  Three  week  post  simulation  scenarios 
(Simulated  Trauma  Encounters  or  STEs)  administered  to  all  participants  (Milestone 
Completion  Date:  30  September  2012) 


17 


Training  was  completed  for  all  participants.  Participants  included  57  medical  hospital  staff 
(emergency  department  nurses  specially  trained  in  trauma  care,  emergency  medical 
technicians,  respiratory  therapists)  and  physicians  (general  surgery  and  emergency  medicine 
residents)  who  worked  in  trauma  teams  in  the  emergency  department  and  the  Level  One 
trauma  center  at  Memorial  Medical  Center.  Third  to  fifth  year  general  surgery  residents  served 
as  team  leaders.  First,  second,  and,  on  occasion,  third  year  residents  served  as  supporting 
physician  members  of  the  teams.  These  individuals  were  randomly  assigned  to  teams  for 
simulated  trauma  resuscitations.  This  method  mirrored  the  way  that  ad-hoc  teams  are  formed  to 
provide  emergency  care  in  the  emergency  department  and  trauma  center.  An  Emergency 
Medical  Technician  (EMT)  who  was  not  a  study  participant  was  trained  to  present  the  case  in  a 
standard  manner.  As  mentioned  earlier,  Figure  1  summarizes  the  revised  design  of  the  study 
and  Table  1  provides  a  description  of  the  behaviors  targeted  for  training. 

The  pretest  simulated  trauma  encounter  (STE)  utilized  high  fidelity  inanimate  computer  driven 
simulators  and  took  approximately  15  minutes  to  complete.  These  pre-intervention  data  were 
used  to  estimate  baseline  performance  The  STEs  were  video  and  audio  recorded  to  allow 
analysis  of  targeted  individual  and  team  behaviors. 

After  the  baseline  STE,  team  members  participated  in  team  training  designed  to  develop  the 
team  leader  and  member  behaviors  chosen  to  improve  team  communication,  leadership  and 
other  role  appropriate  team  behaviors.  The  training  concluded  with  team  practice  with  a  second 
STE  which  was  also  used  to  measure  acquisition  of  target  behaviors.  The  teams  were 
debriefed  after  this  post-training  STE  to  help  cement  desired  behaviors  and  to  correct 
undesirable  behaviors. 

Approximately  three  weeks  later,  participants  were  assigned  to  new  teams,  reassembled,  and 
asked  to  participate  in  a  third,  different,  high  fidelity  STE  to  measure  post-intervention  leader 
and  team  member  behaviors.  These  STEs  were  also  video  and  audio  recorded  for  analysis  and 
followed  by  a  team  debriefing  session.  The  STEs  were  randomly  distributed  amongst  the  pre¬ 
intervention,  immediate  post-intervention  and  three  week  post  intervention  conditions  to  control 
for  possible  differences  in  case  difficulty 


Task  8.  Team  performance  During  Pre,  Immediate  Post  and  Three  Week  Post  Intervention 
STEs  were  analyzed  and  interpreted.  (Milestone  Completion  Date:  31  March 
2012) 

Outcomes  and  Measurements.  We  used  Kirkpatrick’s  Hierarchy  (KH)  of  evaluation  (1)  to  guide 
our  choice  of  outcome  measures.  The  primary  outcome  for  the  study  was  changes  in  individual 
and  team  behaviors  (KH  level  3).  These  changes  were  measured  by  comparing  video  records  of 
pre  intervention,  immediate  post  intervention  and  delayed  post-intervention  STEs.  We 
investigated  coded  frequencies  of  trained  behaviors  and  quality  of  those  performances  using  the 
criteria  specified  in  the  training. 

Secondary  outcomes  included: 

•  Percent  who  completed  training  and  views  on  the  learning  experience,  its  organization, 
presentation,  content,  teaching  methods  (KH  1) 

•  Intention  to  apply  knowledge  and  skills  learned  (KH  3) 

•  Other  attitudes  regarding  the  knowledge  and  skills  learned  (KH  3) 


Raters.  A  surgeon  and  a  nurse  who  were  blinded  to  the  study  group  independently  rated  each 
audio/video  record  Raters  blindly  rated  five  of  the  audio  video  records  twice  with  new  video 
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record  IDs,  which  allowed  us  to  test  within  rater  agreement.  We  trained  the  raters  by  introducing 
and  talking  about  the  rating  form,  which  we  modified  based  on  their  suggestions.  We  then 
reviewed  a  video  and  rated  it  together,  then  viewed  a  second  video  and  allowed  the  raters  to 
rate  it  and  discuss  their  ratings  and  perceptions. 

Data  Analysis.  We  documented  the  impact  on  the  trainees  by  determining  attitudes  toward  the 
program,  clarity  and  nature  of  expressed  intentions  to  change  personal  work  behavior,  and 
clarity  and  nature  of  expressed  intentions  to  advocate  for  changes  in  work  practices  of  the 
department  and  organization.  Additionally  we  analyzed  differences  in  ad-hoc  individual  and 
team  performance  on  the  STEs  in  each  study  group.  The  unit  of  measurement  was  the 
performance  of  the  team  rather  than  that  of  the  individuals  in  the  group. 

Statistical  Analysis.  Within  rater  agreement  was  determined  using  both  the  relative  ranking  and 
absolute  agreement  models  of  intraclass  correlation  coefficients.  The  first  determines  the 
degree  to  which  the  rankings  of  the  targeted  video  were  similar  on  the  first  and  second 
evaluations.  The  second  determines  the  degree  to  which  the  absolute  rating  assigned  on  both 
rating  occasions  agreed.  Attitudes  toward  the  course  and  toward  the  targeted  training  behaviors 
were  analyzed  by  determining  the  number  and  percent  of  respondents  choosing  each  response 
option.  Comparisons  between  pre  training,  immediate  post  training  and  three  week  retention 
video  records  of  team  performances  were  compared  using  one-way  analysis  of  variance. 
Selected  target  behaviors  and  combinations  of  target  behaviors  (e.g.  SMARTT  Stepback 
behaviors)  were  also  singled  out  for  analyses.  In  these  cases  the  frequencies  of  occurrences 
and  percent  of  occasions  where  the  behaviors  occurred  were  determined.  All  analyses  were 
performed  using  Statistical  Package  for  the  Social  Sciences  Software  (SPSS)  version  19  (IBM 
SPSS,  Chicago,  Illinois). 


Results 

Table  2  provides  the  results  of  the  within  rater  agreement  analyses  broken  down  by  rater 
(surgeon,  nurse)  and  by  scale.  The  conventionally  accepted  standard  for  reliability  correlations 
is  0.80  and  above  though  this  standard  is  often  not  obtained  when  using  human  raters. 

Our  findings  demonstrated  that  the  nurse  rater  achieved  this  0.80  standard  of  consistency  in 
most  of  the  measurements  included  in  Table  2  regardless  of  the  model  being  used.  The 
surgeon  rater’s  results  were  lower  than  those  of  the  nurse  rater.  For  subsequent  analyses  we 
combined  the  surgeon  and  nurse  rater’s  ratings. 

Table  3  provides  a  summary  of  participant  responses  to  the  post-training  questionnaire 
regarding  their  perceptions  of  the  training.  Forty  five  participants  (79%)  completed  the  post¬ 
training  questionnaire.  Two  participants  did  not  identify  their  medical  role.  Their  responses  are 
included  in  the  Total  columns.  The  first  six  items  established  participants’  perceptions  about  the 
process  and  content  of  the  training.  The  predominant  response  for  all  items  was  “strongly 
agree”  with  the  exception  of  the  item  regarding  “training  being  a  good  use  of  the  respondents’ 
time”.  Within  these  six  questions,  there  were  six  “undecided”  responses.  Four  of  them  were 
responses  to  the  questions  about  being  a  good  use  of  the  respondents’  time.  The  undecided 
responses  were  fairly  evenly  distributed  among  the  three  groups  of  professionals. 

Questions  7  through  1 1  asked  the  participants  for  their  opinions  regarding  the  potential  benefits 
of  the  target  training  behaviors  for  the  quality  and  efficiency  of  patient  care  and  patient 
outcomes.  For  all  five  questions  the  majority  of  participants  strongly  agreed  that  the  training 
had  the  potential  to  improve  patient  safety,  care  efficiency,  team  functioning,  clarity  regarding 
team  leadership,  better  communication,  situation  awareness,  and  mutual  support.  Item  12 
asked  participants  to  indicate  whether  they  intended  to  apply  the  skills  learned  in  their  work 
environments.  Again  the  predominant  response  was  “strongly  agree”. 
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Across  these  12  items,  physician  respondents  were  more  positive  about  the  benefits  of  the 
training  and  the  potential  quality  and  care  outcomes  than  were  nurses  and  medical  technicians 
although  all  responded  favorably  with  the  exception  of  the  one  nurse  mentioned  earlier.  The 
final  item  on  the  post-training  questionnaire  asked  the  participants  to  indicate  whether  they  had 
already  used  any  or  all  of  the  trained  skills  in  their  work  environment.  Twenty  six  participants 
(58%)  indicated  they  had  used  some  of  the  trained  skills  already.  Two  indicated  they  had  not 
and  16  said  they  had  not  had  an  opportunity  to  use  the  trained  skills  yet  due  to  the  fact  that  at 
the  time  of  training  the  Level  One  trauma  center  was  at  the  other  teaching  hospital  in 
Springfield. 

Table  4  provides  a  summary  of  the  team  results.  Cells  highlighted  in  light  grey  indicate 
statistically  significant  differences  in  team  and  leader  behaviors  that  suggest  the  training  had 
desired  training  effects.  Cells  highlighted  in  dark  grey  indicate  statistically  significant  differences 
in  team  behavior  that  indicate  a  significant  reduction  in  desired  team  and  leader  behaviors  at  the 
three  week  retention  endpoint  compared  to  the  immediate  post-training  endpoint. 

As  can  be  seen  in  Table  4,  14  out  of  17  targeted  team  and  leader  behaviors  significantly 
improved  immediately  following  the  training.  One  area  where  team  and  leader  behaviors  did  not 
improve  included  team  member  efforts  to  clarify  ambiguous  orders.  It  is  not  clear  whether  this 
result  reflects  situations  with  ambiguous  orders  where  team  members  did  not  attempt  to  clarify 
the  ambiguity  or  the  finding  reflects  the  lack  of  ambiguous  orders  and  thus  the  opportunity  and 
need  to  seek  clarity.  We  believe  the  former  is  the  more  accurate  explanation.  The  second  area 
where  improvements  were  not  observed  involved  whether  the  team  leader  was  clearly 
identifiable.  As  can  be  seen  in  the  pre-training  simulation  results,  the  judges’  ratings  indicate 
that  the  team  leader  was  easily  identified  prior  to  training.  This  result  may  indicate  the  lack  of  a 
need  for  training  in  this  specific  area.  The  third  area  where  training  effects  were  not  manifested 
in  the  results  involved  team  leader  management  of  noise.  Anecdotally  observers  independently 
commented  on  the  fact  that  there  was  less  extraneous  noise  in  these  STEs  than  is  normally  true 
in  trauma  events.  This  artifact  may  explain  the  lack  of  significant  differences  observed. 

Comparing  the  pre-training  with  three  week  retention  results  indicates  that  seven  areas  had 
lasting  training  effects.  Moreover,  four  of  these  seven  areas  (efficiency,  listened  to  information, 
orders  were  carried  out,  cooperation  and  communication)  are  all  critical  indicators  of  effective 
team  performance.  One  area  where  there  was  a  critical  relapse  in  team  behavior  was  team 
member  confirmation  when  they  completed  tasks.  This  area  requires  additional  attention. 

Table  5  indicates  the  number  of  team  members  who  introduced  themselves  to  the  scribe  upon 
arrival.  As  can  be  seen  by  inspecting  the  table,  the  number  of  team  members  who  announced 
themselves  before  training  was  virtually  zero  and  this  was  true  regardless  of  profession.  The 
rate  of  introductions  improved  immediately  after  training  and  was  maintained  at  a  slightly  lower 
rate  after  three  weeks.  However  the  rate  is  still  not  100%. 

Table  6  summarizes  the  number  of  teams  where  at  least  one  coordinated,  complete  SMARTT 
Stepback  occurred  during  the  training  simulation.  For  this  Table  we  recorded  that  a  SMARTT 
Stepback  occurred  only  if  all  six  components  occurred  at  a  single  time.  As  can  be  seen  the 
frequency  of  SMARTT  Stepbacks  increased  after  training  and  the  rate  was  sustained  after  three 
weeks.  However  the  rate  was  still  less  than  the  desired  100%.  Table  7  indicates  the  number  of 
SMARTT  Stepback  component  behaviors  that  occurred  at  some  point  during  each  simulation,  a 
much  less  restrictive  indication  of  training  success.  In  this  table  a  frequency  of  9  in  a  cell  would 
indicate  that  this  behavior  occurred  one  time  in  that  simulation  event.  As  can  be  seen,  the  two 
raters  appeared  to  use  a  different  rating  strategy  for  the  pretraining.  Their  frequencies  for  the 
immediate  post-training  and  three  weeks  retention  post  training  are  more  similar.  Raters 


recorded  that  each  element  occurred  close  to  once  per  simulation  event  except  in  the  pre¬ 
training  phase  of  the  study. 
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Discussion 

The  in-situ  simulation  that  we  conducted  showed  that  residents  were  not  clear  on  who  was 
leading  resuscitations  and  this  resulted  in  a  shifting  leadership  focus  among  the  residents 
throughout  the  trauma  resuscitation.  Communication  was  fragmented,  incomplete,  and 
frequently  interrupted  thus  requiring  repetition.  The  trauma  bay  was  noisy  with  several  people 
often  talking  at  once.  These  findings  were  confirmed  subsequently  in  observations  of  actual 
trauma  resuscitations  in  the  trauma  bay. 

Our  training  intervention  was  developed  specifically  to  determine  whether  a  brief  training 
program  would  lead  to  changes  in  team  member  behaviors  that  would  improve  team 
performance  in  these  areas. 

Recently  two  other  studies  have  addressed  similar  training  needs  for  ad  hoc  trauma  team 
members  (2,  3).  Both  of  those  training  interventions  involved  more  training  time  on  the  part  of 
participants,  compared  with  our  study.  Likewise  both  studies  included  changes  in  traditional 
trauma  outcome  parameters  as  well  as  changes  in  team  behaviors  whereas  our  study  focused 
exclusively  on  changes  in  targeted  team  behaviors  manifested  in  simulated  trauma 
resuscitations.  Finally,  both  studies  included  attending  physicians  as  participants  while  our  study 
excluded  attending  physicians.  The  primary  value  that  our  study  adds  to  the  findings  of  these 
two  studies  lies  in  two  areas.  First  and  perhaps  most  important,  our  study  used  expert  judges 
who  were  blinded  to  the  stage  of  training  for  participants.  Our  raters  reviewed  audio-video 
records  of  all  teams  performing  all  trauma  simulations  at  every  stage  of  training.  The  audio¬ 
video  records  were  randomly  ordered  and  the  rater  did  not  know  the  training  stage  for  the 
performing  team.  Second,  our  study  added  the  three  week  post-training  simulation  to  measure 
team  retention  of  targeted  team  and  individual  behaviors.  All  three  studies  demonstrated  that 
relatively  brief  training  episodes  can  lead  to  changes  in  targeted  team  leadership, 
communication  and  coordination  behaviors. 

In  our  study  all  behaviors  were  observed  in  STEs  where  participants  knew  what  behaviors  were 
being  observed  and  recorded  which  may  raise  a  question  about  whether  the  trained  behaviors 
will  persist  in  the  actual  trauma  environment  with  real  patients.  Flowever,  the  results  from  the 
Capella  et  al  (3)  and  Steinemann  et  al  (3)  studies  provide  some  evidence  to  support  transfer  of 
training  to  real  trauma  cases.  Our  study  does  strengthen  the  collective  knowledge  from  the 
three  studies  by  blinding  the  raters  to  stage  of  training  and  thus  minimizing  the  possibility  that 
raters’  judgments  are  influenced  by  their  knowledge  of  the  training  stage.  Our  results  are  also 
conservative  in  that  the  retention  outcome  measure  occurs  prior  to  the  impact  of  the  second 
debriefing. 

While  the  results  of  our  study  provide  evidence  that  the  training  produced  intended  results,  it  is 
clear  that  the  effect  was  not  robust.  The  targeted  behaviors  were  not  present  in  all  teams  and 
the  sporadic  team  behavioral  characteristics  remind  us  that  the  behaviors  are  likely  to  fade 
absent  practice  and  continued  hospital  leadership  support  in  the  form  of  policies  and  role 
modeling.  For  example  inspection  of  Tables  6  and  7  indicate  that,  at  most,  two  thirds  of  the 
team  leaders  initiated  a  complete  SMARTT  Stepback  event  during  the  simulations  occurring 
after  training.  Flowever  many  of  the  elements  of  a  SMARTT  Stepback  occurred  during  each  post 
training  simulation.  This  suggests  partial  success  in  this  portion  of  the  training  but  certainly  also 
indicates  that  more  training  and  reinforcement  of  these  coordinated  behaviors  will  be  needed  if 
SMARTT  Stepbacks  are  expected  to  occur  on  a  regular  basis. 
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All  team  leaders  and  prospective  team  leaders  were  trained  and  were  given  opportunity  to 
practice  these  behaviors  as  a  part  of  this  project.  A  nucleus  group  of  nursing  and  emergency 
department  technicians  also  received  the  training  including  the  practice  simulated  trauma 
resuscitations.  All  other  trauma  team  members  are  receiving  the  didactic  training  but  will  not 
have  the  opportunity  to  practice  using  the  simulated  trauma  resuscitations.  The  trauma 
leadership,  hospital  leadership  and  emergency  medicine  leadership  are  committed  to 
encouraging  all  team  members  to  incorporate  these  behaviors  into  their  individual  and  team 
practices  by  providing  mandatory  training  to  the  remaining  trauma  nurses  and  technicians  and 
to  the  new  general  surgery  and  emergency  department  residents. 

Finally  we  want  to  echo  the  views  of  the  investigators  in  the  Capella  et  al  study(3)  that  the 
logistics  of  providing  training  to  teams  made  up  of  health  care  providers  from  different 
professions  was  one  of  the  most  challenging  aspects  of  this  project.  Coordinating  the 
scheduling  of  these  training  exercises  involved  working  with  five  different  hospital  and  residency 
program  administrative  bodies.  It  is  no  surprise  to  us  that  there  is  a  great  deal  of  talk  about  the 
desirability  of  multi-professional  team  training  but  the  number  of  examples  of  such  training  in 
hospital  settings  is  limited. 


Key  Research  Accomplishments 

•  Respondents  strongly  agreed  that  the  process  and  content  of  the  training  was 
worthwhile. 

•  Respondents  strongly  agreed  that  the  training  had  the  potential  to  improve  patient 
safety,  care  efficiency,  team  functioning,  clarity  regarding  team  leadership,  better 
communication,  situation  awareness,  and  mutual  support. 

•  Twenty  six  participants  (58%)  indicated  they  had  used  some  of  the  trained  skills  already. 
Two  indicated  they  had  not  and  16  said  they  had  not  had  an  opportunity  to  use  the 
trained  skills  yet  due  to  the  fact  that,  at  the  time  of  training,  the  Level  One  trauma  center 
was  at  the  other  teaching  hospital  in  Springfield. 

•  Physician  respondents  were  more  positive  about  the  benefits  of  the  training  and  the 
potential  quality  and  care  outcomes  than  were  nurses  and  medical  technicians  although 
all  but  one  responded  favorably. 

•  14  out  of  17  targeted  team  and  leader  behaviors  significantly  improved  immediately 
following  the  training.  One  area  where  team  and  leader  behaviors  did  not  improve 
included  team  member  efforts  to  clarify  ambiguous  orders.  The  second  area  where 
improvements  were  not  observed  involved  whether  the  team  leader  was  clearly 
identifiable.  As  can  be  seen  in  the  pre-training  simulation  results,  the  judges’  ratings 
indicate  that  the  team  leader  was  easily  identified  prior  to  training.  This  result  may 
indicate  the  lack  of  a  need  for  training  in  this  specific  area. 

•  The  third  area  where  training  effects  were  not  manifested  in  the  results  involved  team 
leader  management  of  noise.  Anecdotally,  observers  independently  commented  on  the 
fact  that  there  was  less  extraneous  noise  in  these  simulated  trauma  events  than  is 
normally  true  in  trauma  settings.  This  artifact  may  explain  the  lack  of  significant 
differences  observed. 

•  Three  week  retention  results  indicate  that  seven  areas  had  lasting  training  effects. 
Moreover,  four  of  these  seven  areas  (efficiency,  listened  to  information,  orders  were 
carried  out,  cooperation  and  communication)  are  all  critical  indicators  of  effective  team 
performance.  One  area  where  there  was  a  critical  relapse  in  team  behavior  was  team 
member  confirmation  when  they  completed  tasks.  This  area  requires  additional 
attention. 


The  frequency  of  complete  SMARTT  Stepback  events  increased  after  training  and  the 
rate  was  sustained  after  three  weeks.  However  the  rate  was  still  less  than  the  desired 
100%.  The  frequency  of  SMARTT  Stepback  elements  used  in  isolation  did  improve. 
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Reportable  Outcomes 

Aspects  of  our  findings  have  been  presented  in  poster  form  at  the: 

•  Association  of  American  Medical  Colleges  Central  Group  on  Educational  Affairs  Annual 
Meetings  held  in  Saint  Louis,  Missouri  on  March  29-31, 2012 

•  Emergency  Medicine  Council  of  Emergency  Medicine  Residency  Program  Directors 
Annual  Meetings  held  in  Atlanta,  Georgia  on  April  1  -  April  4,  2012 

•  American  Association  for  the  Surgery  of  Trauma  Annual  Meetings  held  from  September 
12-15,  2012  in  in  Kauai,  Hawaii. 

•  A  paper  has  been  submitted  for  Presentation  at  the  201 3  Annual  Meetings  of  the 
Association  for  Surgical  Education  which  is  a  joint  meeting  with  the  Association  of 
Program  Directors  in  Surgery.  A  manuscript  describing  this  research  and  the  findings 
has  been  prepared  and  will  be  submitted  for  consideration  to  be  published  in  the 
American  Journal  of  Surgery. 


Conclusion 

While  the  results  of  our  study  provide  evidence  that  the  training  is  producing  intended  results,  it 
is  clear  that  the  effects  are  not  strong  enough  to  instill  confidence  that  the  behaviors  will  be 
sustained  without  practice  and  continued  hospital  leadership  support  in  the  form  of  policies, 
procedures  and  role  modeling.  The  targeted  behaviors  are  not  present  in  all  teams  and  are 
sporadic  in  others.  For  example  inspection  of  Tables  6  and  7  indicate  that,  at  most,  two  thirds  of 
the  team  leaders  initiated  a  complete  SMARTT  Stepback  event  during  the  simulations  occurring 
after  training.  However  many  of  the  elements  of  a  SMARTT  Stepback  occurred  during  each  post 
training  simulation.  This  suggests  partial  success  in  this  portion  of  the  training  but  certainly  also 
indicates  that  more  training  and  reinforcement  of  these  coordinated  behaviors  will  be  needed  if 
SMARTT  Stepbacks  are  expected  to  occur  on  a  regular  basis. 

All  team  leaders  and  prospective  team  leaders  were  trained  and  were  given  opportunity  to 
practice  these  behaviors  as  a  part  of  this  project.  A  nucleus  group  of  nursing  and  emergency 
department  technicians  also  received  the  training  including  the  practice  simulated  trauma 
resuscitations.  All  other  trauma  team  members  are  receiving  the  didactic  training  but  will  not 
have  the  opportunity  to  practice  using  the  simulated  trauma  resuscitations.  The  trauma 
leadership,  hospital  leadership  and  emergency  medicine  leadership  are  committed  to 
encouraging  all  team  members  to  incorporate  these  behaviors  into  their  individual  and  team 
practices  by  providing  mandatory  training  to  the  remaining  trauma  nurses  and  technicians  and 
to  the  new  general  surgery  and  emergency  department  residents. 

Finally  we  want  to  echo  the  views  of  the  investigators  in  the  Capella  et  al  study(3)  that  the 
logistics  of  providing  training  to  teams  made  up  of  health  care  providers  from  different 
professions  was  one  of  the  most  challenging  aspects  of  this  project.  Coordinating  the 
scheduling  of  these  training  exercises  involved  working  with  five  different  hospital  and  residency 
program  administrative  bodies.  It  is  no  surprise  to  us  that  there  is  a  great  deal  of  talk  about  the 
desirability  of  multi-professional  team  training  but  the  number  of  examples  of  such  training  in 
hospital  settings  is  limited. 

All  revised  milestones  have  been  achieved.  All  key  players  from  the  emergency  department  and 
the  trauma  center  played  active  roles  in  the  final  design  and  development  of  the  training  and 
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training  materials.  They  also  played  key,  visible  roles  in  delivering  the  training  and  debriefing  the 
teams  after  simulated  trauma  resuscitations.  Further,  while  outside  the  goals  of  this  contract,  the 
didactic  portion  of  this  training  has  now  been  delivered  to  97%  of  health  care  professionals  who 
provide  trauma  care  at  Memorial  Medical  Center  with  the  goal  of  training  them  all.  All  new 
general  surgery  and  emergency  medicine  residents  have  also  received  the  training  and 
previously  trained  residents  also  participated  in  these  sessions  as  a  form  of  refresher  training. 
The  Director  of  the  Trauma  Center  has  informed  all  health  care  personnel  that  his  expectations 
are  that  the  trained  procedures  will  be  used  routinely  in  all  trauma  care  situations  in  this 
hospital.  The  project  has  been  finished  ahead  of  the  revised  schedule  and  below  budget. 
Memorial  Medical  Center  provided  $6,647.23  in  unreimbursed,  in-kind  personnel  costs  in 
support  of  this  project. 
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T able  1 .  T earn  Behaviors  T argeted  for  T raining 


Team  Leader  Behaviors 

Decide  what  needs  to  be  done  and  priorities 

Direct  orders  to  specific  people  by  name  and  or  role  (e.g.  nurse,  respiratory 
technician) 

Issue  short,  clear  orders 

Explicit  transitions  in  team  leadership 

Manage  noise  in  the  trauma  bay 

Manage  workload  to  achieve  team  balance 

Ensure  that  team  members  adhere  to  orders  and  protocol 

Encourage  all  members  to  volunteer  key  information 

Team  Member  Behaviors 

Introduce  self  by  name  and  function  to  the  scribe  and  other  team  members 

Carry  out  orders  issued  by  the  managing  physician  and  standing  orders 
appropriate  to  role 

Communicate  critical  information  to  all  members  of  the  team 

Speak  only  when  necessary  for  patient  care.  Minimize  unnecessary 
noise/distraction/talk 

Listen  to  information  provided  by  other  team  members 

Verbally  confirm  orders  and  completion  of  tasks 

Seek  clarification  regarding  who  is  the  managing  physician  when  ambiguity 
exists 

Communication  Behaviors  -  SMARTT 

Stepback  (Led  by  team  leader) 

Situation  (Patient  description,  injury,  status,  circumstances) 

Management  (Treatment  performed) 

Activity  (What  needs  to  happen  next?) 

Rapidity  (What  needs  to  be  done  first  and  how  quickly?) 

Troubleshoot  (What  may  go  wrong  and  steps  to  correct  or  prevent) 

Talk  to  Me  (Encourage  all  team  members  to  volunteer  key  information,  ask 
clarifying  questions,  etc). 
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Table  2.  Within  Rater  Agreement  for  Five  Repeated  Cases  (Blindly  Rated) 


Within  Rater  Agreement  (Intra-class  correlation  coefficient) 

Scale 

Surgeon  Rater 

Nurse  Rater 

Relative 

Absolute 

Relative 

Absolute 

Ranking 

Agreement 

Ranking 

Agreement 

Professionals  Announced 

0.12 

0.12 

0.84 

0.85 

SMARTT  Step  Back  (Dichotomous  Items) 

0.50 

0.50 

0.84 

0.83 

SMARTT  Step  Back  (Quality  Scale) 

0.60 

0.57 

0.94 

0.94 

Team  Behavior  Scale 

0.74 

0.74 

0.85 

0.78 

Team  Leadership  Scale 

0.54 

0.54 

0.79 

0.77 

Trauma  NoTechs  Scale 

0.60 

0.58 

0.74 

0.75 

Average 

0.52 

0.51 

0.83 

0.82 
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Table  3.  Participant  views  on  the  learning  experience,  broken  down  by  medical  role  of  respondents  (One  nurse 


respondent  who  responded  “strongly  disagree”  to  all  items  is  excluded  from  the  table  to  simplify  the  table) 
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Table  4.  Mean  ratings  (and  standard  deviations)  of  targeted  and  trained  ad-hoc  trauma  team  behaviors  during 
simulated  trauma  resuscitations  at  each  stage  in  the  team  training  cycle. 


Category  (Best  Possible 

Score) 

Pretraining 

Immediate 

Post 

Training 

3  Week 

Retention 

Pretraining 

vs 

Immediate 

Post 

Training 

Pretraining 

vs  3  wk 

Retention 

Immediate 

Post 

Training 

vs  3  wk 

Retention 

Mean  (SD) 

Mean  (SD) 

Mean  (SD) 

P 

P 

P 

Organized  (5) 

2.67  (0.66) 

3.67  (0.35) 

3.67  (0.50) 

0.001 

0.001 

NS 

Efficiency  (5) 

3.00  (0.67) 

3.78  (0.36) 

3.78  (0.51) 

0.01 

0.01 

NS 

Supportive  (5) 

3.00  (0.50) 

3.67  (0.25) 

3.44  (0.46) 

0.007 

NS 

NS 

Volunteered  Important 

Information  (5) 

2.61  (0.55) 

3.89  (0.22) 

3.28  (0.69) 

0.00 

NS 

NS 

Listened  to  Information  (5) 

3.06  (0.58) 

3.83  (0.25) 

3.67  (0.25) 

0.001 

0.008 

NS 

Instructions  performed  (5) 

3.17(0.35) 

3.78  (0.26) 

3.61  (0.33) 

0.001 

0.018 

NS 

Clarified  ambiguous 

orders  (5) 

1.89  (0.82) 

3.00  (1.20) 

2.50(1.04) 

NS 

NS 

NS 

Confirmed  completion  of 

tasks  (5) 

1.89  (0.33) 

3.06  (0.68) 

2.39  (0.55) 

0.00 

NS 

0.03 

Leader  clearly  identifiable 

(5) 

3.78  (0.51) 

4.0  (0) 

3.83  (0.35) 

NS 

NS 

NS 

Leader  assigned  tasks  to 

team  members  by  name 

or  role  (5) 

3.11  (0.65) 

3.89(0.22) 

3.28  (0.67) 

0.02 

NS 

NS 

Leader  managed  noise 

2.41  (1.20) 

3.50  (0.84) 

3.43  (0.79) 

NS 

NS 

NS 
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Category  (Best  Possible 

Score) 

Pretraining 

Immediate 

Post 

Training 

3  Week 

Retention 

Pretraining 

vs 

Immediate 

Post 

Training 

Pretraining 

vs  3  wk 

Retention 

Immediate 

Post 

Training 

vs  3  wk 

Retention 

Mean  (SD) 

Mean  (SD) 

Mean  (SD) 

P 

P 

P 

appropriately  (5) 

Leader  coordinated 

communication  and  team 

activity  effectively  (5) 

2.33  (0.75) 

3.44  (0.46) 

2.72  (0.67) 

0.003 

NS 

NS 

Leader  encouraged  team 

members  to  volunteer  key 

information  during 

SMARTT  Step-Back  (5) 

1.50  (0.71) 

3.58  (0.58) 

3.25(1.17) 

0.04 

NS 

NS 

TRAUMA  NOTECHS 

Leadership  (5) 

3.72  (0.36) 

4.67  (0.43) 

4.22  (0.67) 

0.002 

NS 

NS 

Cooperation  (5) 

2.89  (0.65) 

4.61  (0.42) 

3.94  (0.92) 

0.000 

0.01 

NS 

Communication  (5) 

2.56  (0.46) 

4.06  (0.39) 

3.50  (0.97) 

0.000 

0.015 

NS 

Decision  Making  (5) 

3.67  (0.71) 

4.61  (0.70) 

4.17(0.97) 

0.05 

NS 

NS 

Situation  Awareness 

Coping  with  Stress  (5) 

3.67  (0.71) 

4.67  (0.35) 

4.39  (0.70) 

0.005 

0.05 

NS 

3' 


Table  5.  Team  Members  Who  Were  Announced  upon  Arrival  Broken  Down  by  Profession  and  by  Reporting 
Rater 


31 


Table  6.  Number  (Percent)  of  Simulations  where  a  SMARTT  Stepback  Occurred  as  a  Coordinated  Event 
(All  components  occurred  at  same  time)  Broken  Down  by  Time  and  by  Rater 


Time 

Rater  One  -  Frequency  (Percent) 

Rater  Two  -  Frequency  (Percent) 

Pre  Training 

0(0) 

0(0) 

Immediate  Post  Training 

3(33) 

6(67) 

Three  Weeks  Post  Training 

5(56) 

6(67) 
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Table  7.  SMARTT  Stepback  Components  that  Occurred  During  the  Simulation  Broken  Down  by  Time  and 
By  Rater  (A  single  performance  which  included  one  instance  of  the  behavior  would  have  a  frequency  of  9  in 
each  cell) 


Time 

Rater 

Situation 

Management 

Activity 

Rapidity 

Troubleshoot 

Talk  to 

Me 

Pre 

Training 

1 

8 

8 

7 

2 

2 

6 

2 

0 

0 

0 

0 

0 

0 

Immediate 

Post 

Training 

1 

9 

9 

9 

8 

8 

9 

2 

6 

6 

6 

6 

5 

5 

Three 

Weeks  Post 

Training 

1 

8 

8 

7 

6 

4 

6 

2 

7 

8 

8 

5 

6 

7 
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Appendix  1 

Protocol  for  Debriefing  Participants  After  Practice  Trauma 

Resuscitation  Sessions 

The  training  intervention  for  members  of  the  trauma  team  is  targeted  toward  changing  the  SMARTT 
Stepback,  Communication,  Leadership  and  Teamwork  Behaviors  described  on  the  following  pages.  After 
the  training  session,  team  members  will  be  assigned  to  a  trauma  team  and  asked  to  resuscitate  a  patient 
while  focusing  on  incorporating  the  behaviors  targeted  in  the  training  session.  The  trauma  resuscitation 
will  take  approximately  10-15  minutes.  Immediately  following  the  trauma  resuscitation  practice  session, 
the  team  members  will  sit  down  with  you  to  discuss  the  trauma  resuscitation  experience. 

Tell  the  team  members  you  are  going  to  play  the  video  of  the  team’s  performance.  You  should  initiate 
discussion  along  the  way  to  discuss  elements  of  the  team’s  performance  either  at  your  initiation  or  upon 
request  from  a  team  member. 

Tell  the  team  members  that  the  goals  of  the  debriefing  session  are  to  provide  an  opportunity  to  discuss: 

•  what  targeted  training  behaviors  worked 

•  what  training  behaviors  didn’t  work,  or  were  hard  to  implement 

•  how  to  make  that  aspect  work  better  in  the  future 

•  Remind  them  that  the  focus  of  the  conversation  should  be  on  the  SMARTT  Stepback, 
Communication,  Leadership  and  Teamwork  Behaviors  learned  during  the  training  not  clinical 
care 

You  should  also  offer  your  own  observations  and  suggestions  about  the  team’s  performance  including: 

•  Identifying  positive  SMARTT  Stepback,  Communication,  Leadership  and  Teamwork  Behaviors 
exhibited  by  the  team. 

•  Gaps  in  team  performance  related  to  the  target  behaviors 

•  Specific  suggestions  on  how  to  change  those  behaviors  the  next  time 

Wrap  up  the  debriefing  by 

•  Reinforcing  key  learning  points  identified  by  learners 

•  Adding  learning  points  of  your  own  (Take  home  messages) 

•  Providing  a  few  specific  suggestions  that  can  be  applied  to  future  trauma  team  practice. 

We  think  the  debriefing  will  be  most  productive  if  you: 

•  Focus  on  the  behavior  of  the  group  not  on  the  clinical  performances  of  individuals 

•  Create  a  psychologically  safe  environment  for  participants 

•  Make  sure  all  participants  actively  contribute  to  the  discussion  and  self-reflect  upon  what 
happened 

•  Use  a  non-threatening  but  honest  approach  understanding  some  learners  may  be  unhappy  with 
the  training 

•  Give  specific  suggestions  and  relate  to  trauma  team  experience. 

•  Keep  the  debriefing  as  short  as  possible 


Supporting  Data 
Meeting  Abstracts 
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Nicole  K.  Roberts,  PhD,  John  Sutyak,  MD,  Christopher  McDowell,  MD,  David  Griffen,  MD, 
Jarrod  Wall,  MD,  Cathy  Schwind,  RN,  MS,  Reed  G.  Williams,  PhD,  Be  SMARTT about  Trauma: 
An  Interdisciplinary  Educational  Intervention  for  the  Trauma  Bay,  presented  at  the  Association 
of  American  Medical  Colleges  Central  Group  on  Educational  Affairs  Annual  Meetings  held  in 
Saint  Louis,  Missouri  on  March  29-31, 2012. 

Background:  Patient  care  outcomes  are  adversely  affected  by  poor  teamwork  and 
communication.  Trauma  teams,  which  tend  to  be  constituted  on  an  ad  hoc  basis,  are 
particularly  prone  to  communication  problems  and  an  absence  of  overt  leadership  behaviors. 
This  project  was  developed  to  improve  leadership  behaviors  and  communication  in  ad  hoc 
teams. 

Method:  We  performed  and  videotaped  an  in-situ  trauma  simulation  to  explore  how  trauma 
resuscitations  were  currently  performed.  A  team  of  trauma  physicians,  Emergency  Medicine 
physicians  and  educators  reviewed  the  video  and  analyzed  the  problems  with  teamwork  and 
communication  exposed  there.  The  in-situ  observation  was  augmented  by  observations  in  the 
trauma  bay.  Based  on  the  observations  and  discussion  with  trauma  and  emergency  room 
personnel  (including  nurses,  technicians  and  physicians)  we  developed  an  interprofessional 
educational  intervention  to  address  deficits  in  teamwork  and  communication. 

We  delineated  desired  team  behaviors  and  team  leader  behaviors.  We  also  created  a 
structured  approach  to  communication  in  the  trauma  bay.  Emergency  Medicine  and  Trauma 
surgeons  taught  the  behaviors  to  the  Trauma  health  care  team:  EM  and  Surgery  residents,  EM 
techs,  Trauma  nurses,  and  Respiratory  therapists.  The  communication  module,  called  the 
SMARTT  Stepback,  consists  of  the  following:  Situation,  Management,  Activity,  Rapidity, 
Troubleshooting,  Talk  to  Me.  This  approach  to  communication  ensures  that  all  members  of  the 
health  care  team  have  a  shared  understanding  of  the  patient’s  status,  and  have  a  controlled 
opportunity  to  offer  their  observations.  The  training  session  consisted  of  a  pre-test  simulated 
trauma  resuscitation,  a  brief  didactic  session  delivered  by  a  Trauma  surgeon,  and  a  post¬ 
training  simulated  trauma  resuscitation  where  participants  were  allowed  to  practice  the  skills 
they  learned.  The  post  training  session  was  followed  by  a  debriefing  by  Trauma  surgeon  or  an 
Emergency  Medicine  physician 

Results:  Forty  four  of  49  participants  (92%)  filled  out  a  brief  evaluation  of  the  educational 
intervention.  Of  special  note,  all  participants  (except  one  nurse  who  marked  Strongly  Disagree 
for  all  statements)  agreed  or  strongly  agreed  that  using  the  model  could  lead  to  improved  care 
efficiency,  improved  patient  safety,  and  better  communication,  situation  awareness,  and  mutual 
support.  Participants  noted  that  the  debriefing  was  especially  effective  in  opening  lines  of 
communication  among  the  professions.  Observations  of  debriefing  revealed  discussions 
focused  on  creating  and  maintaining  role  clarity  and  on  seeing  the  value  of  all  team  members. 


Conclusions:  Deliberate  education  on  team  behavior,  leadership,  and  a  structured  model  for 
communication  that  ensures  shared  understanding  among  all  team  members  is  likely  to  lead  to 
better  patient  care. 


35 


Christopher  McDowell  MD,  Nicole  K.  Roberts  PhD,  John  Sutyak  MD,  David  Griffen  MD  PhD, 
Jarrod  Wall  MD,  Cathy  Schwind  RN  MS,  Reed  G.  Williams  PhD,  Be  SMARTT  About  Trauma: 
An  Interdisciplinary  Educational  Approach  to  Improving  Teamwork  in  the  Trauma  Bay, 
presented  at  Emergency  Medicine  Council  of  Emergency  Medicine  Residency  Program 
Directors  Annual  Meetings  held  in  Atlanta,  Georgia  on  April  1  -  April  4,  2012. 


Background:  Patient  care  outcomes  are  adversely  affected  by  poor  teamwork.  Trauma  teams 
with  their  ad  hoc  constitution  are  particularly  prone  to  communication  problems  and  an  absence 
of  overt  leadership  behaviors. 

Objective:  This  project  was  developed  to  improve  leadership  behaviors  and  communication  in 
ad  hoc  trauma  teams. 

Method:  We  performed  and  videotaped  an  in-situ  trauma  simulation  to  explore  how  trauma 
resuscitations  were  currently  performed.  This  simulation  was  augmented  by  observations  of 
actual  trauma  resuscitations.  A  team  of  trauma  surgeons,  EM  physicians  and  educators 
reviewed  the  video  and  analyzed  the  exposed  problems  with  teamwork  and  communication. 
Based  on  the  observations  and  discussion  with  trauma  team  members,  we  developed  an 
interprofessional  educational  intervention  to  address  deficits  in  teamwork  and  communication. 
We  delineated  desired  team  behaviors  and  team  leader  behaviors.  We  also  created  a 
structured  approach  to  communication  in  the  trauma  bay.  EM  physicians  and  Trauma  surgeons 
taught  the  behaviors  to  the  Trauma  team:  EM  and  Surgery  residents,  EM  techs,  Trauma  nurses, 
and  Respiratory  therapists.  The  communication  module,  called  the  SMARTT  Stepback,  consists 
of  the  following:  Situation,  Management,  Activity,  Rapidity,  Troubleshooting,  Talk  to  Me.  The 
training  consisted  of  a  pre-test  simulated  trauma  resuscitation,  a  brief  didactic  session  delivered 
by  a  Trauma  surgeon,  and  a  post-training  simulated  trauma  resuscitation  where  participants 
were  allowed  to  practice  the  skills  they  learned.  The  post  training  session  was  followed  by  a 
debriefing  by  Trauma  surgeon  or  an  EM  physician. 

Results:  44  of  49  participants  (92%)  evaluated  the  educational  intervention.  All  participants 
(except  one  nurse  who  marked  Strongly  Disagree  for  all  statements)  agreed  or  strongly  agreed 
that  using  the  SMARTT  model  could  improve  efficiency,  patient  safety,  communication,  situation 
awareness,  and  mutual  support.  Participants  noted  the  debriefing  was  especially  effective  in 
opening  lines  of  communication  among  the  professions.  Debriefing  discussions  focused  on 
creating  and  maintaining  role  clarity  and  on  seeing  the  value  of  all  team  members. 

Conclusions:  Deliberate  education  on  team  behavior,  leadership,  and  a  structured  model  for 
communication  that  ensures  shared  understanding  among  all  team  members  is  likely  to  lead  to 
better  patient  care. 
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Jarrod  Wall,  Christopher  D.  Wohltmann*,  M.D.,  Hilary  Sanfey,  M.D.,  Audra  Chestnut,  Reed 
Williams,  Ph.D.,  Southern  Illinois  University  School  of  Medicine,  SMARTT  Team  Training 
Improves  Ad-Hoc  Trauma  Team  Dynamics,  presented  at  American  Association  for  the  Surgery 
of  Trauma  Annual  Meetings  held  from  September  12-15,  2012  in  Kauai,  Hawaii. 


Introduction:  Trauma  teams  frequently  form  ad  hoc,  with  limited  familiarity 
among  members,  creating  a  milieu  for  communication  lapses.  This  study 
created  &  analyzed  a  brief  communication  and  leadership  intervention. 
Methods:  Analysis  of  simulated  &  real  team  activities  revealed  gaps  in 
organization,  efficiency  &  information  exchange.  The  SMARTT  Step  Back 
(Situation,  Management,  Activity,  Rapidity,  Troubleshoot,  Talk  to  me) 
communication  protocol  was  created  for  information  exchange.  PGY  3-5  GS 
residents  lead  9  teams  with  PGY  1-2  GS  &  EM  residents,  ED  RN’s,  EMT’s  & 
RT’s.  Each  group  attended  a  didactic  program  between  pre  and  post-test 
simulations.  Retention  post  test  occurred  at  3  wks.  Two  independent  blinded 
raters  scored  videos  using  a  5  point  Likert  scale  covering  surgical  NOTECHS 
items  &  other  team  activity.  49  participants  were  own  controls.  Statistical 
analysis:  ANOVA,  t-test,  Chi  square,  significance  p<0.05. 

Results:  43  of  44  responding  participants 
agreed  SMARTT  improved  efficiency, 
safety,  communication,  awareness  and 
support,  Team  improvements  &  SMARTT 
usage  were  maintained  over  3  weeks 
(p=NS  for  3  wks  vs.  Post  1). 

Conclusions:  Teams  significantly 
improved  on  a  majority  of  aspects, 
including:  communication,  mutual 
support,  information  transfer,  &  leadership 
following  a  brief  training  activity. 

Institution  of  the  SMARTT  Step-Back  is 
easy  &  significantly  improves  ad  hoc  team 
communication. 


Category 

Postl 
v  Pre 

3 

Wks 
v.  Pre 

Organized 

<0.001 

0.001 

Efficiency 

0.005 

0.013 

Supportive 

0.01 

0.12 

Listened  to  Info 

0.005 

0.03 

Instructions  Performed 

0.003 

0.04 

NOTECHS 

Leadership 

<0.01 

0.26 

Cooperation 

<0.001 

0.02 

Communication 

<0.001 

0.02 

Decision  Making 

0.09 

NS 

Awareness/Stress 

0.02 

0.12 
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Griffen,  Jarrod  Wall,  Hilary  Sanfey,  Audra  Chestnut,  Andreas  Meier,  Christopher  Wohltmann,  Ted  R.  Clark, 
Nathaniel  Wetter-T aylor,  The  impact  of  brief  team  communication,  leadership  and  team  behavior  training 
on  ad  hoc  team  performance  in  trauma  care  settings,  to  be  presented  at  the  201 3  Annual  Meetings  of  the 
Association  for  Surgical  Education. 


Abstract 


Background 

Communication  breakdowns  and  care  coordination  problems  often  cause  preventable  adverse  patient  care 
events,  which  can  be  especially  acute  in  the  trauma  setting  where  ad  hoc  teams  have  little  time  for 
advanced  planning.  Existing  teamwork  curricula  do  not  address  the  particular  issues  associated  with  ad 
hoc  emergency  teams  providing  trauma  care. 

Methods 

Ad  hoc  trauma  teams  completed  a  pre-instruction  simulated  trauma  encounter  (STE)  and  were  provided 
instruction  on  appropriate  team  behaviors  and  team  communication.  Teams  completed  a  post-instruction 
STE  immediately  and  three  weeks  later,  then  completed  a  questionnaire.  Blinded  raters  rated  videotapes  of 
the  simulations 

Results 

Participants  expressed  high  levels  of  satisfaction  and  intent  to  change  practice  after  the  intervention. 
Participants  changed  teamwork  and  communication  behavior  in  the  post  test  and  changes  were  sustained 
after  a  three  week  interval,  though  there  was  some  loss  of  retention.  See  table  1 . 

Conclusions 

Brief  training  exercises  can  change  teamwork  and  communication  behaviors  in  ad  hoc  trauma  teams. 
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